1 - 



PATENT ABSTRACTS OF JAPAN 



JP200 1-277 163 



* NOTICES * 

JPO and EVPIT are not responsible for anydamages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



(51)Int.Cl. 



(21) Application number : 2000-100795 

(22) Date of filing : 03.04.2000 



(1 l)Publication number : 2001-277163 
(43)Date of publication of application : 09.10.2001 
B25J 9/22 
B25J 5/00 
B25J 13/00 
G05B 13/02 
G05B 19/4155 
G06N 3/00 

(71) Applicant : SONY CORP 

(72) Inventor : KUROKI YOSHIHIRO 
ISHTOA KENZO 



(54) DEVICE ANDMETHOD FOR CONTROLLING ROBOT 

~~ llEl(ti] (57)Abstract: 

9 | PROBLEM TO BE SOLVED: To investigate a cause even if 
abnormality or a failure occurs or a trouble happens during 
'interactive driving' or 'autonomous driving'. 
SOLUTION: A user for providing a command to a robot can be 
certified. The face and voice of the user that cannot be certified 
or information on the other living body's characteristic is 
extracted. Command content provided from the user is recorded 
together with an action of the robot in response to the command 
or an execution time of the action. An internal state of the robot 
and sensor input information may be recorded. In the future, the 
^recorded contents are analyzed to investigate a cause of 
^ |abnormality or a failure, or a trouble occurring for the robot. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the robot which operates according to an action plan and 
relates to the action plan mold robot which forms an action plan by the dialogue with the user based on the 
input of voice, an image, etc. especially, or does not depend on a user input but forms an action plan 
autonomously. 

[0002] Furthermore, in detail, this invention generates abnormalities and failure during the period which 
performs "a dialogue drive" or "an autonomous drive", or even if it is the case where it is involved in a 
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trouble, it relates to the action plan mold robot which can study the cause 
[0003] 

[Description of the Prior Art] The thing of the machinery which performs movement modeled on actuation 
of human being using the electric or magnetic operation is called "robot." It is said that a robot's origin of a 
word originates in "ROBOTA (slave machine)" of a slab word. Although it was in our country that a robot 
began to spread from the end of the 1960s, the many were the industrial robots (industrial robot) in works 
aiming at automation, full automation, etc. of production, such as a manipulator and a carrier robot. 
[0004] Like an arm type robot, the robot of a deferment type which is implanted and used for a certain 
specific location works only in fixed and local workspaces, such as assembly, a sorting activity etc of 
components. On the other hand, the robot of workspace of a portable type is un-restrictive, and he can move 
free in a predetermined path or non-path top, and the human activity of predetermined or arbitration can be 
executed by proxy, or he can offer the various broad services which replace Homo sapiens, a dog or other 
life objects. Especially, the mobile robot of a leg formula compares with a crawler type and tire-type robot 
and although it is unstable and attitude control and walk control become difficult, he is excellent in the point 
that flexible walk / transit actuation which a stairway, rise and fall of a ladder, and an obstruction get over 
and does not ask distinction of ** and the leveling and the irregular ground is realizable. 
[0005] Recently, the researches and developments about leg formula mobile robots, such as a robot 
(humanoid robot) of "the human form" designed by using as a model the body mechanism of an animal and 
actuation which perform the pet mold robot which imitated the body mechanism of the animal of 
quadrapedalism and its actuation like a dog or a cat, or a 2-pair-of-shoes walk in erect posture like Homo 
sapiens, or "a human mold", progress, and the expectation for utilization has also been growing 
[0006] As one of the applications of a leg formula mobile robot, vicarious execution of various kinds of 
difficulty activities in an industrial activity, a production activity, etc. is mentioned. For example it is 
vicarious execution of the maintenance in a nuclear power plant, a thermal power station plant and a 
petrochemical plant, conveyance and assembly operation of the components in a plant, cleaning in a 
skyscraper, and the risk activity and difficulty activity like the rescue in a fire site and others etc 
[0007] Moreover, the application of "symbiosis" with a life adhesion mold, i.e., human being, is mentioned 
rather than above-mentioned activity exchange as other applications of a leg formula mobile robot. This kind 
of robot emulates the mechanism of leg formula ambulatory exercises with comparatively high intelligence 
such as Homo sapiens or a dog (pet), of operation, and a feeling expression. Moreover, it is required it not ' 
only performs faithfully the pattern of operation inputted beforehand, but that it should realize the lively 
expression of operation in response to a partner's language and attitudes ("it strikes"). [ "it praises" or "he 
scolding", ] 

[0008] The conventional toy machine has the fixed relation between user actuation and response actuation 

ChangC aCtUati ° n ° f a t0y accordin S to "king of a user. Consequently, a user becomes 
****************** soon about the tQy wh . ch repeats Qnly game acmation 

[0009] On the other hand, an intellectual robot performs intellectual actuation according to the time series 
model of generation of operation while performing autonomous thinking and motion control Moreover 
when a robot equips a picture input device and voice-input/output equipment and performs an image 
processing and speech processing, it also becomes possible to realize realistic communication with human 
being on more advanced intellectual level. Under the present circumstances, by answering having detected 
the stimulus from the outside, such as user actuation, and changing this time series model, namely giving the 
"study effectiveness", or it does not get bored for a user, the pattern of operation which was adapted for 
[0010] Can bC ° ffered ' Moreover ' a user can en J°y a kind of training simulation like a game. 

[Problem(s) to be Solved by the Invention] What is necessary is just to perform from a user or an operator by 
inputting almost uniquely the command which can be interpreted in the case of the robot for specified uses 
on industry. This is similar to an information management system answering that the console input was 
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carried out, and operating the most important commands, such as the copy and deletion of a file and file 
opening, just like that. 

[001 1] on the other hand, in the case of the robot of a symbiosis mold or an entertainment mold "an 
autonomous drive" which operates according to the action plan which the robot itself formed can be 
performed, without performing "a dialogue drive" which operates based on high input data of whenever 
[ abstract ], such as not only the explicit or restrictive command from a user but voice, and an image 
comparatively, or depending on a command and a dialogue from a user further, as mentioned above (namely 
— from a user - becoming independent). 

[0012] However, when interpret an abstract user input, a robot's original thing thinking control is permitted 
or a part, and the failure and the trouble which the degree of freedom and function on actuation develop are 
caused, the cause investigation becomes much more difficult. 

[0013] Since the response by the side of the equipment to an input command corresponds to about 1 to 1 in 
the case of a general machinery, when which command is inputted or executed, it can specify easily whether 
abnormalities occurred as equipment. 

[0014] On the other hand, since width of face is in the interpretation on the robot to a user input or an 
external event in the case of the robot of a type which performs above "dialogue drives" or "autonomous 
drives , a judgment becomes difficult about in what the cause of abnormalities or failure originates If it is 
furthermore a leg formula mobile robot, since he will walk in non-path and it will turn around the workspace 
of arbitration, a user cannot monitor continuously. Therefore, it will become very difficult for a user to study 
what happened to the robot then, even if a certain failure occurred or it was involved in the trouble accident 
or an incident, etc. under the situation of having been placed out of a user's monitor 
[0015] The purpose of this invention is to offer the great action plan mold robot which can form an action 
plan by the dialogue with the user based on the input of voice, an image, etc., or cannot depend on a user 
input but can form an action plan autonomously. 

[0016] The further purpose of this invention generates abnormalities and failure during the period which 
performs dialogue drive" or "an autonomous drive", or even if it is the case where it is involved in a trouble 
it is to offer the great action plan mold robot which can study the cause 
[0017] 

[Means for Solving the Problem] This invention is made in consideration of the above-mentioned technical 
problem, and consists of two or more joint actuators. The action plan section or the step which is control 
device or approach of the robot of a type which operates according to an action plan, and forms said robot's 
action plan, The control section of operation or step which realizes the pattern of operation corresponding to 
the action plan determined by said action plan section or step by the drive of each joint actuator The 
detecting element or step which detects the implementation situation of operation by said control section of 
operation or step, It is a robot's control unit or approach characterized by providing the Records Department 
or the step which records a log including the implementation situation of operation by said action plan 
section, the action plan by the step and said detecting element, or the step. 

[0018] A robot's control device or approach concerning this invention may also contain the dialogue 
Management Department or the step which manages a dialogue with the user based on the user input 
command or data which minded further the user input section or the step which receives the command or 
data from a user, and said user input section or step. In such a case, said action plan section or step can form 
an action plan according to the contents of a dialogue in said dialogue Management Department or step, and 
said Records Department or step can take the log of the contents of a dialogue 

[00 19] Moreover, the self-test section or the step which diagnoses said robot's each part may also be included 
further. In such a case, said Records Department or step can take the log of a self-test result 
[0020] Moreover, the user authentication section which carries out authentication processing of the user who 
is around said robot may also be included further. In such a case, said Records Department or step can take 
the log of a user authentication result. 
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[0021] Moreover, said action plan section or step operates with the 2nd method of operation which forms an 
action plan based on the feeling model determined according to the 1st method of operation which forms an 
action plan based on a dialogue with a user, or external change. In such a case, said Records Department or 
step can take the log of the method of operation in the action plan section or a step 
[0022] 

[Function] The robot concerning this invention can search the workspace of arbitration in non-path like a leg 
formula mobile robot. This workspace is sharable with human being's dwelling space. Moreover, the robot 
concerning this invention can form an action plan by the dialogue with the user based on the input of voice, 
an image, etc., or further, cannot depend on a user input but can form an action plan autonomously. 
[0023] The user who gives a robot a command can be attested. Or the description information on a user's 
face which cannot be attested, or voice or other living bodies is extracted. Furthermore, it records as a kind 
of log combining the contents of the command given by the user, and the activation time of day of action or 
action which the robot took according to the command. In addition, a robot's internal state and sensor input 
may be recorded collectively. 

[0024] The cause of the abnormalities and failure which took place to the robot, and a trouble can be studied 
by analyzing the contents of the log later. 

[0025] Moreover, at the time of abnormalities, even if it is not at the trouble time at the time of failure the 
hysteresis of a robot's action of a robot, such as "who and what have been carried out where", can be ' 
investigated. Or the hysteresis of action can be managed as experience information, and it can constitute so 
that the information "who and what have been carried out where" can be pulled out out of a dialogue with a 
user and a robot. A user can enjoy a kind of training simulation like a game highly [ such a robot / 
entertainment orientation ]. 

[0026] The purpose, the description, and advantage of further others of this invention will become [ rather 
than ] clear by detailed explanation based on the example and the drawing to attach of this invention 
mentioned later. 
[0027] 

[Embodiment of the Invention] Hereafter, the example of this invention is explained in detail, referring to a 
drawing. 

[0028] Signs that signs that the leg formula mobile robot 100 with which operation of this invention is 
presented was standing straight were viewed from each of the front and back are shown in drawing 1 and 
drawm ? 2 • this le S formula mobile robot 100 is a type called "a human form" or "a human mold" a dialogue 
with the user based on the input of voice, an image, etc. can stand an action plan, or it cannot depend on a 
user input but a robot (namely, - from a user - becoming independent) 100 can form an action plan 
autonomously so that it may mention later. The leg formula mobile robot 100 consists of the membrum 
infenus of two pairs of right and left which perform leg formula migration, the truncus section an upper 
extremity on either side, and a head. 

[0029] the membrum inferius of each right and left consists of a femoral region, a knee joint, a leg part an 
ankle, and ****, and is connected by the hip joint at the abbreviation lowest edge of the truncus section 
moreover, the upper extremity of each right and left consists of an overarm, an elbow joint, and a forearm 
and is connected by the shoulder joint in right-and-left each side edge of the truncus section upper part 
Moreover, the head is connected in the center of the abbreviation best edge of the truncus section by the neck 
joint. 

[0030] In the truncus section unit, the control section which is not visible is arranged on d rawin g I and 
dmwin«2 . This control section is the controller (main control section) which processes the external input 
from drive control of each joint actuator, each sensor (after-mentioned), etc. which constitutes the leg 
formula mobile robot 100, and a case carrying the peripheral devices of a power circuit and others In 
addition to this, the control section may contain the communication link interface and communication device 
for remote operation. 
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[0031] The joint degree-of-freedom configuration which the leg formula mobile robot 100 concerning this 
example possesses is typically shown in drawing 3 . The leg formula mobile robot 100 consists of the truncus 
sections which connect the upper part of the body including two arms and a head 1, the membrum inferius 
which consists of the two legs which realize migration actuation, and an upper extremity and the membrum 
inferius as illustration. 

[0032] The neck joint which supports a head 1 has three degrees of freedom called the neck joint yawing axis 
2, the neck joint pitching axis 3, and the neck joint roll axes 4. 

[0033] Moreover, each arm consists of the shoulder-joint pitching axis 8, the shoulder -joint roll axes 9 the 
overarm yawing axis 10, the elbow-joint pitching axis 1 1, the forearm yawing axis 12, a wrist joint pitching 
axis 13, wrist joint roll axes 14, and a hand part 15. Hand parts 15 are the many joints and the 
multi-degree-of-freedom structure containing two or more fingers in fact. However, since there are little the 
contribution and effect to posture stability control or walk motion control of a robot 100, the actuation of a 
hand part 1 5 itself assumes on these specifications that it is a zero degree of freedom. Therefore, each arm on 
either side presupposes that it has seven degrees of freedom. 

[0034] Moreover, the truncus section has three degrees of freedom called the truncus pitching axis 5, the 
truncus roll axes 6, and the truncus yawing axis 7. 

[0035] moreover, the leg of each right and left which constitute the membrum inferius consists of the hip 
joint yawing axis 16, the hip joint pitching axis 17, the hip joint roll axes 18, the knee-joint pitching axis 19 
an ankle joint pitching axis 20, joint roll axes 21, and a foot (vola or ****) 22. In fact, although the foot 
(vola) 22 of the body is the structure containing the vola of many joints and many degrees of freedom, the 
vola of the leg formula mobile robot 100 concerning this example makes it a zero degree of freedom. 
Therefore, each leg on either side consists of six degrees of freedom. 

[0036] If the above is summarized, as the leg formula mobile-robot 100 whole concerning this example it 
will have 2= 3+7x2+3+6x32 degree of freedom in total. However, the leg formula mobile robot 100 is not 
necessarily limited to 32 degrees of freedom. It cannot be overemphasized that a degree of freedom i e the 
number of joints, can be suitably fluctuated according to a constraint, requirement specification, etc. on a 
design / manufacture. 

[0037] Each above-mentioned joint degree of freedom which the leg formula mobile robot 100 has is 
realized as active actuation by the actuator in fact. As for various requests, such as eliminating an excessive 
swelling by the exterior of equipment and making it approximate in the shape of [ human ] a natural bodily 
shape, or performing attitude control to the unstable structure called a 2-pair-of -shoes walk to a joint 
actuator, it is desirable that it is small and lightweight. In this example, we decided to carry' the small AC 
servo actuator of the type which was a gear direct attachment type, and one-chip-ized the servo control 
system and was built in the motor unit. In addition, it is indicated about the small AC servo actuator 
applicable to a leg formula robot by the Japanese-Patent-Application-No. No. 33386 [ 1 1 to ] specification 
already transferred, for example to these people. 

[0038] The control-system configuration of the leg formula mobile robot 100 concerning this example is 
typically shown in dj^winjLi • As shown in this drawing, this system consists of a thinking control module 
200 which responds to a user input etc. dynamically and manages emotion decision and a feeling expression 
and a movement control module 300 which controls a robot's whole body coordinated movements, such as a 
drive of a joint actuator. 

[0039] The thinking control modules 200 are CPU (Central Processing Unit)21 1 which performs data 
processing about emotion decision or a feeling expression, and an independent information processor which 
consists of RAM (Random Access Memory)212, ROM (Read Only Memory)213, and external storage (hard 
disk drive etc.) 214 and which can perform self-conclusion processing. 

[0040] Various kinds of equipments, such as the communication link interface 254 which performs the 
system and the data exchange besides a robot 100, are connected to the thinking control module 200 by the 
bus interface 201 course via the audio output devices 253, such as the picture input devices 251 such as a 
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CCD (Charge Coupled Device) camera, the audio input units 252, such as a microphone, and a loudspeaker, 
LAN (not shown [ Local Area Network : ]), etc. 

[0041] In the thinking control module 200, the acoustic-sense data inputted from the vision data inputted 
from a picture input device 251 or an audio input unit 252 determine the leg formula mobile robot's 100 
current feeling and current intention according to the stimulus from the external world etc. Furthermore a 
command is published to the movement control module 300 so that the behavior or action based on decision 
making, i.e., movement of the limbs, may be performed. 

[0042] One movement control modules 300 are CPU (Central Processing Unit)31 1 which controls a robot's 
100 whole body coordinated movement, and an independent information processor which consists of RAM 
(Random Access Memory)3 1 2, ROM (Read Only Memory)313, and external storage (hard disk drive etc ) 
314 and which can perform self-conclusion processing. The action plan of the walk pattern computed 
off-line, a ZMP target orbit, and others can be accumulated in external storage 314 ("ZMP" is a point on the 
floor line where the moment by the floor reaction force during a walk serves as zero, and a "ZMP orbit" 
means the locus to which ZMP moves during a robot's 100 walk actuation period etc.). 
[0043] various kinds of equipments , such as power control which manage the power source of the 
touch-down check sensors 352 and 353 which detect the bed leaving of the joint actuator ( refer to drawing 
3 ) which realize each joint degree of freedom distribute to a robot 100 whole body , the attitude sensor 35 1 
which measure the posture and inclination of the truncus section , and the vola on either side , or 
implantation , a dc-battery , etc. , be connect to the movement control module 300 by the bus interface 301 
course . 



[0044] The whole body coordinated movement according the action directed from the thinking control 
module 200 to body present ****** and each joint actuator is controlled by the movement control module 
300. That is, CPU311 picks out the pattern of operation according to the action directed from the thinking 
control module 200 from external storage 3 14, or generates a pattern of operation internally, and the pattern 
of operation with which CPU3 1 1 was specified - following - a foot - while setting up movement, a ZMP 
(Zero Moment Point) orbit, truncus movement, upper extremity movement, a lumbar part horizontal position 
height, etc., the command value which directs actuation according to these contents of a setting is transmitted 
to each joint actuator. 

[0045] Moreover, CPU31 1 can control the leg formula mobile robot's 100 whole body coordinated 
movement accommodative by detecting whether each movable foot is in which condition of **** or a basis 
with the output signal of each touch-down check sensors 352 and 353 while detecting the posture and 
inclination of a truncus part of a robot 100 with the output signal of an attitude sensor 351 
[0046] Furthermore, the action as the intention that the movement control module 300 was determined in the 
thinking control module 200 returns how many body present ******, i.e., the situations of processing to the 
thinking control module 200. 

[0047] The thinking control module 200 and the movement control module 300 are built on a common 
platform, and it interconnects through bus interfaces 201 and 301 between both. 

[0048] The leg formula mobile robot 100 concerning this example records combining the contents of the 
command given by the user, and the activation time of day of action or action which the robot took according 
to the command. Therefore, a user can study the cause of the abnormalities and failure which took place to 
the robot, and a trouble by analyzing the contents of record later. Moreover, at the time of abnormalities 
even ,f it is not at the trouble time at the time of failure, the hysteresis of a robot's action of a robot, such as 
who and what have been carried out where", can be investigated. Below, recording processing of a robot's 
action hysteresis in this invention or experience information is explained. 

[0049] In drawings , an example of the functional-block configuration for realizing record processing of 
action record or experience information in the leg formula mobile robot 100 is illustrated The robot 100 
which shows in this drawing can operate the "dialogue drive" method which follows a dialogue with the user 
based on the input of voice, an image, etc., and forms an action plan. 



11 - 



JP200 1-277 163 

[0050] The leg formula mobile robot 100 uses at least one side among the fingerprint sensor 1 1 1 or a visual 
sensor 1 13 for user authentication. 

[0051] A visual sensor 1 13 can make the picture input devices 251, such as a CCD (Charge Coupled Device- 
charge-coupled device) camera carried in the head, serve a double purpose. In this case, the description 
information extracted from one or more registered users 1 face image, the image which projected other parts 
on a user's body, or the image is kept beforehand. And the authentication processing section 1 14 carries out 
authentication processing of whether you are a registered user by carrying out comparison collating with the 
description information, such as a face image registered beforehand, while carrying out the image processing 
of the face image inputted from the visual sensor 113. 

[0052] Moreover, although the fingerprint sensor 1 1 1 is not illustrated to drawing 1 - especially drawing 4 
it arranges the fingerprint read head in the part on which users, such as a head and a shoulder, tend to put a' 
fingertip, for example. And one or more registered users' fingerprint is beforehand kept as authentication 
information. And the authentication processing section 1 12 carries out authentication processing of whether 
you are a registered user by carrying out comparison collating with the fingerprint registered beforehand 
while an image processing, feature-extraction processing, etc. carry out fingerprint information inputted from 
the fingerprint sensor 1 1 1 . In addition, JP.3-18980.A is equipped with the criteria image memory holding the 
image data of the fingerprint used as criteria, and the picture input device which inputs the image of the 
fingerprint used as the candidate for authentication, and it is indicated about the fingerprint collation device 
of the type which performs fingerprint authentication by piling up both images and seeing whenever 
[ coincidence ]. 

[0053] The fingerprint sensor 1 1 1 and the authentication processing section 1 12 can perform more exact 
authentication processing to the user who demanded fingerprint authentication clearly. On the other hand 
even if a visual sensor 1 13 and especially the authentication processing section 1 14 are the cases where a' 
user does not require, a user's figure can be pursued with a camera and a user can also perform authentication 
processing to the inside of unconscious. Moreover, the authentication processing sections 1 12 and 1 14 can 
transmit a user's authentication result to the information recording section 125. 

[0054] The leg formula mobile robot 100 concerning this example offers four kinds of means of the speech 
recognition section 115, the communications control section 116, the image recognition section 1 17 and a 
commander 1 18, in order that a user may input a command and data. 

[0055] The speech recognition section 1 15 is constituted by CPU21 1 (or other data-processing circuits) in 
which the voice input sections 252, such as a microphone, and speech recognition processing are possible A 
user can input a natural language format or an abstract command, such as "run", "carry O**" and "hurry" 
with voice. The speech recognition section 1 15 recognizes a user's voice, and the command interpretation' 
section 1 19 understands and analyzes the command to a robot 100 based on a speech recognition result 
further. 

[0056] Moreover, the communications control section 1 16 is constituted by CPU21 1 (or other 
data-processing circuits) which performs processing of the command or data exchanged for the 
communication link interface 254 by communication link interface 254 course. The communication link 
interface 254 ,s the wireless data communication network course of bluetooth or others, and is an external 
information terminal (for example, a personal computer etc.). It interconnects not illustrating. On such an 
information terminal, automatized machines, such as a robot, can carry out the console input of the command 
of the format which can be interpreted, or the command of a format more abstract than it uniquely and a user 
can transmit to a robot 100 side. After the user command by which reception was carried out in the 
communications control section 1 16 is interpreted in the command interpretation section 119, it is 
transmitted to the dialogue Management Department 1 24. 

[0057] The image recognition section 1 1 7 is constituted by CPU21 1 (or other data-processing circuits) in 
which the .mage input sections 25 1, such as a CCD camera, and image recognition processing are possible 
A user can mput a command by the gesture, a gesture, gesture, etc. The image recognition of the image 
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recognition section 1 17 is carried out, further, the command interpretation section 1 19 is based on a 
recognition result, and it understands and analyzes it as a command. Or you may make it input a command 
by carrying out the image input of the check-by-looking information, such as a cyber-code uniquely assigned 
to the command instead of gesture. 

[0058] Although a commander 1 18 does not illustrate to drawing 1 - especially drawing 4 , he is constituted 
as the user input equipment which was carried in the leg formula mobile robot's 100 the back, an abdomen, 
etc. and in which a key/button grabbing is possible, or a remote controller (not shown). As for a user, 
automatized machines, such as a robot, can drive in uniquely the command of the format which can be 
interpreted, or the command of a format more abstract than it on a commander 118. After the user command 
inputted on the commander 1 18 is interpreted in the command interpretation section 1 19, it is transmitted to 
the dialogue Management Department 124. 

[0059] The dialogue Management Department 124 is managing serially the command of the various formats 
received by the command interpretation section 1 19 course, and it aims at maintenance of a dialogue context 
while it understands the context of a dialogue with a user. In this example, the dialogue Management 
Department 124 can transmit the contents of a dialogue with a user to the information recording section 125 
Moreover, the dialogue Management Department 124 can do a voice output outside by the loudspeaker, after 
generating the reply according to a user input in the speech synthesis section 120, or it can perform a console 
output (or image output using GUI) on an external computing system (for example, user terminal) by 
communications control section 121 course. 

[0060] The speech synthesis section 120 is constituted by the audio output devices 253, such as a 
loudspeaker, and CPU21 1 (or other data-processing circuits) in which speech synthesis processing is 
possible. Moreover, the communications control section 121 is constituted by CPU21 1 (or other 
data-processing circuits) which performs processing of the command or data exchanged for the 
communication link interface 254 by communication link interface 254 course. 

[0061] An attitude sensor 130, a temperature sensor 131, the joint angle sensor 132, a touch sensor 133 a 
force sensor 134, the power-source Management Department 135, and the communications control detecting 
element 136 are the functional modules for detecting a robot's 100 the external world or environment, and 
the change and an activation situation of operation. 

[0062] An attitude sensor 130 is equivalent to the reference number 354 in drawing 4 , and can detect the 
posture over a robot's 100 road surface. Moreover, the joint angle sensor 132 is equivalent to the circular 
encoder arranged m each joint actuator (refer to drawing 3 ) of every. Moreover, a touch sensor 133 is 
equivalent to the touch-down check sensor 352,352 arranged by the foot. Moreover, although a force sensor 
134 is not illustrated to drawing 1 - drawing 4 , it is arranged by at least each part of a robot's 100 whole 
body, and detects contact with the obstruction of a user or the external world (for example, indicated by the 
Japanese-Patent-Application-No. No. 366175 [ 1 1 to ] specification already transferred to these people about 
the sensor module which can be carried in a robot's each part of the whole body in the form of exterior parts) 
[0063] The power-source Management Department 135 is equivalent to the reference number 354 in drawing 
4 , and supervises the supply voltage of the dc-battery which is a robot's 100 main power supply, a current a 
power supply, etc. Moreover, the communications control detecting element 136 is constituted by CPU21 1 
(or other data-processing circuits) which performs processing of the command or data exchanged for the 
communication link interface 254 by communication link interface 254 course, and detects change of the 
external world by the communication link command or commo data, or an environment. 
[0064] A robot's 100 environmental factor specified by these sensors is supplied to the action plan section 
1 27 and the self-test section 1 29. 

[0065] According to the context of the dialogue in the dialogue Management Department 124, a robot's 100 
the external world or an environment specified in each sensor ability module 130 -, the action plan section 
127 forms a robot's 100 action plan, and orders implementation of action to the control section 128 of 
operation. Moreover, in this example, the action plan section 1 27 can transmit the contents of the action plan 
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to the information recording section 125. 

[0066] The control section 128 of operation publishes operating commands, such as a rotation command and 
each rate command, to each joint actuator (refer to drawing 3 and drawing 4 ) that action directed by the 
action plan section 127 should be realized. Consequently, a robot's 100 action planned or expected is realized 
The rotation location of each joint actuator is measured by each joint angle sensor 132, and can perform 
feedback control. 

[0067] Although the control section 128 of operation may generate dynamically the pattern of operation for 
realizing action ordered serially on real time, it may carry out easy [ of the trajectory plan of the main 
patterns of a walk or others of operation ] (namely, off-line count) beforehand. If an action instruction is 
published from the action plan section 127 in the case of the latter, the control section 128 of operation calls 
the trajectory plan of the pattern of operation applicable to this, and it can realize action, correcting a target 
orbit suitably. In addition, by using for JP,62-97006, A the walk pattern data memorized beforehand, while 
simplifying a control program, it is indicated about the multi-joint bipedal robot control unit which can 
connect between each data of a walk pattern densely. 

[0068] The self-test section 129 carries out the self-test of the situation of the robot 100 interior for the 
activation situation of the planned action, abnormalities, failure, a trouble, etc. based on the detection output 
of sensors, such as an attitude sensor 130, a temperature sensor 131, the joint angle sensor 132 a touch 
sensor 133, a force sensor 134, the power-source Management Department 135, and the communications 
control detecting element 136. Moreover, in this example, the self-test section 129 can transmit a self-test 
result to the information recording section 125. 

[0069] The information recording section 125 can acquire the following matters from each part. 

(1) The activation situation of the action plan diagnosed in the action plan (4) self-test section 129 
determined as the contents of a dialogue, the newest contents of user command (3) dialogue or the user 
command managed at the authentication processing section 1 12 and/or the user authentication information 

(2) dialogue Management Department 124 from 1 14 by change of the external world or an environment etc 
[0070] And the information recording section 125 combines these data, makes them together with the current 
time further supplied from a clock 126, and is saved as "action hysteresis" or "experience information" i e 
an action log. 

[0071] An action log is saved in un-volatilizing on the local memory or the local disk (not shown) of the 
information recording section 125. Or if the action log is stored in the store in which insert and remove are 
possible by cartridge-type like a memory stick, it can remove from a robot 100 later and analysis processing 
of the action log can be carried out on an external computing system. 

[0072] Moreover, the "action hysteresis" or the "experience information" saved in the information recording 
section 125 can be retrieved using time information. Or a dialogue with the robot 100 through the speech 
recognition section 1 15 etc. can draw out. 

[0073] In drawing 6 , other examples about the functional-block configuration for realizing record processing 
of action record or experience information in the leg formula mobile robot 100 are illustrated. The robot 100 
which shows in this drawing can operate the "autonomous drive" method which does not depend on the 
dialogue with a user other than the "dialogue drive" method which follows a dialogue with the user based on 
the input of voice, an image, etc., and forms an action plan, but forms an action plan autonomously further 
[0074] The leg formula mobile robot 100 uses at least one side among the fingerprint sensor 1 1 1 or a visual 
sensor 1 13 for user authentication. A visual sensor 1 13 can make the picture input devices 251, such as a 
CCD (Charge Coupled Device: charge-coupled device) camera carried in the head, serve a double purpose 
Moreover, although the fingerprint sensor 1 1 1 is not illustrated to drawing I - especially drawing 4 it 
arranges the fingerprint read head in the part on which users, such as a head and a shoulder tend to put a 
fingertip, for example. 

[0075] The fingerprint sensor 1 1 1 and the authentication processing section 1 12 can perform more exact 
authentication processing to the user who demanded fingerprint authentication clearly. On the other hand, 
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even if a visual sensor 1 13 and especially the authentication processing section 1 14 are the cases where a 
user does not require, a user's figure is pursued with a camera and a user performs authentication processing 
to the inside of unconscious. Moreover, the authentication processing sections 1 12 and 1 14 transmit a user's 
authentication result to the information recording section 125. 

[0076] Moreover, the leg formula mobile robot 100 offers four kinds of means of the speech recognition 
section 1 15, the communications control section 1 16, the image recognition section 1 1 7, and a commander 
1 1 8, in order that a user may input a command. 

[0077] The speech recognition section 1 15 is constituted by the audio input units 252, such as a microphone 
and CPU211 (or other data-processing circuits) in which speech recognition processing is possible. If a user' 
inputs with voice, the speech recognition section 115 will recognize a user's voice, and the command 
interpretation section 1 19 will understand and analyze the command to a robot 100 based on a speech 
recognition result further. 

[0078] Moreover, the communications control section 1 16 is constituted by CPU21 1 (or other 
data-processing circuits) which performs processing of the communication link interface 254, and a 
communication link command or data. The communication link interface 254 interconnects with the external 
information terminal (for example, personal computer) via a wireless data communication network The 
communications control section 1 16 carries out reception of the command which a user transmits via a 
communication line. After the received command is interpreted in the command interpretation section 119, it 
is transmitted to the dialogue Management Department 124. 

[0079] The image recognition section 1 17 is constituted by CPU21 1 (or other data-processing circuits) in 
which the image input sections 251, such as a CCD camera, and image recognition processing are possible 
If, as for a user, a user command is inputted in the form of a gesture, a gesture, gesture, etc the image 
recognition section 1 17 carries out the image recognition of this, and further, the command interpretation 
section 1 19 will be based on a recognition result, and it will understand and analyze it as a command. 
[0080] A commander 1 18 is constituted as the user input equipment which was carried on the leg formula 
mobile robot 100 and in which a key/button grabbing is possible, or a remote controller (not shown) As for a 
user, a robot can drive in uniquely the command of the format which can be interpreted etc. on a commander 
118. After the user command inputted on the commander 1 18 is interpreted in the command interpretation 
section 1 1 9, it is transmitted to the dialogue Management Department 1 24. 

[0081] The dialogue Management Department 124 is managing serially the command of the various formats 
received by the command interpretation section 119 course, and it aims at maintenance of a dialogue context 
while it understands the context of a dialogue with a user. The dialogue Management Department 124 
transmits the contents of a dialogue with a user to the information recording section 125. Moreover the 
dialogue Management Department 124 generates the reply according to a user input in the speech synthesis 
section 120, and through a loudspeaker, a voice output is carried out outside or it outputs the contents of a 
dialogue to it on an external computing system by communications control section 121 course. 
[0082] The speech synthesis section 120 is constituted by the audio output devices 253 such as a 
loudspeaker, and CPU21 1 (or other data-processing circuits) in which speech synthesis processing is 
possible. Moreover, the communications control section 121 is constituted by CPU21 1 (or other 
data-processing circuits) which performs processing of the command or data exchanged for the 
communication link interface 254 by communication link interface 254 course. 

[0083] An attitude sensor 130, a temperature sensor 131, the joint angle sensor 132, a touch sensor 133 a 
force sensor 134, the power-source Management Department 135, and the communications control detecting 
element 136 are the functional modules for detecting a robot's 100 the external world or environment, and its 
change. 

[0084] An attitude sensor 130 detects the posture over a robot's 100 road surface. Moreover the joint angle 
sensor 132 is equivalent to the circular encoder arranged for every joint actuator. Moreover, a touch sensor 
133 is equivalent to the touch-down check sensor 352,352 arranged by the foot. Moreover, a force sensor 
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134 is arranged by at least each part of a robot's 100 whole body, and detects contact with the obstruction of 
a user or the external world. The power-source Management Department 135 supervises the supply voltage 
of the dc-battery which is a robot's 100 main power supply, a current, a power supply, etc. Moreover, the 
communications control detecting element 136 is constituted by CPU21 1 (or other data-processing circuits) 
which performs processing of the communication link interface 254, and a communication link command or 
data, and detects change of the external world by the communication link command or commo data, or an 
environment. 

[0085] A robot's 100 environmental factor specified by these sensors is supplied to the autonomous action 
plan section 140 and the self-test section 129. 

[0086] Based on change of the external environment based on the input data from the above-mentioned 
sensors etc., the autonomous action plan section 140 is determining the feeling model in the feeling model 
condition machine 141, forms a robot's 100 action plan autonomously, and orders implementation of action 
to the control section 128 of operation. Moreover, the autonomous action plan section 140 can transmit the 
contents of the action plan to the information recording section 125. (It is indicated about the leg formula 
robot which it has a feeling instinct model resulting from actuation on the Japanese-Patent-Application-No. 
No. 341374 [ 1 1 to ] specifications already transferred to these people, and a feeling instinct model is 
changed in addition to them based on input, and controls actuation.) 

[0087] The control section 128 of operation publishes operating commands, such as a rotation command and 
each rate command, to each joint actuator that action directed by the autonomous action plan section 140 
should be realized. Consequently, a robot's 100 action planned or expected is realized. The rotation location 
of each joint actuator is measured by each joint angle sensor 132, and can perform feedback control. 
[0088] Although the control section 128 of operation may generate dynamically the pattern of operation for 
realizing action ordered serially on real time, it may carry out easy [ of the trajectory plan of the main 
patterns of a walk or others of operation ] beforehand. 

[0089] The self-test section 129 carries out the self-test of the situation of the robot 100 interior for the 
activation situation of the planned action, abnormalities, failure, a trouble, etc. based on the detection output 
of sensors, such as an attitude sensor 130, a temperature sensor 131, the joint angle sensor 132, a touch 
sensor 133, a force sensor 134, the power-source Management Department 135, and the communications 
control detecting element 136. Moreover, the self-test section 129 transmits a self-test result to the 
information recording section 125. 

[0090] The information recording section 125 can acquire the following matters from each part. 

(1) The activation situation of the action plan diagnosed in the action plan (4) self-test section 129 which the 

autonomous action plan section 140 determined according to the contents of a dialogue or the newest user 

command (3) feeling model managed at the authentication processing section 1 12 and/or the user 

authentication information (2) dialogue Management Department 124 from 1 14 [0091] And the information 

recording section 125 combines these data, makes them together with the current time further supplied from 

a clock 126, and is saved as "action hysteresis" or "experience information", i.e., an action log. 

[0092] An action log is saved in un-volatilizing on the local memory or the local disk (not shown) of the 

information recording section 125. Or if the action log is stored in the store in which insert and remove are 

possible by cartridge-type like a memory stick, it can remove from a robot later and analysis processing of 

the action log can be carried out on an external computing system. 

[0093] Moreover, when supporting the "dialogue drive" method with which the single robot 100 follows a 
dialogue with the user based on the input of voice, an image, etc., and forms an action plan, and the method 
of the both sides of an "autonomous drive" method which do not depend on a dialogue with a user but form 
an action plan autonomously of operation, you may make it an action plan record collectively the 
identification information of what is depended on any of a dialogue drive or an autonomous drive. 
[0094] The "action hysteresis" or the "experience information" saved in the information recording section 
125 can be retrieved using time information. Or a dialogue with the robot 100 through the speech recognition 
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section 1 15 etc. can draw out. 

[0095] The data format of an action log is illustrated in drawing 7 . As shown in this drawing, one log 
consists of the time-of-day field, user authentication information field, the contents field of a dialogue, the 
action plan field, the mode-of-operation field, and an action result and self-test result field. 
[0096] The real time supplied from a clock 126 is written in the time-of-day field. Moreover, User 
Information attested by the fingerprint sensor 1 1 1 and the authentication processing section 1 12 or a visual 
sensor 113, and the authentication processing section 1 14 is written in user authentication information field 
An authentication user may be the command issue origin on dialogue actuation. Moreover, the contents of a 
dialogue managed at the dialogue Management Department 124 are written in the contents field of a 
dialogue. 

[0097] Moreover, the action plan formed by the action plan section 127 or the autonomous action plan 
section 140 is written in the action plan field. Moreover, the information which identifies whether the current 
robot's 100 action plan was determined by which method of a dialogue drive or an autonomous drive of 
operation is written in the mode-of-operation field. 

[0098] Moreover, the implementation situation of operation computed based on the output of each joint 
angle sensor 132 and the self-test results (failure situation etc.) judged based on each other sensor outputs are 
written in an action result and self-test result field. 

[0099] Please fully understand the point that a robot's 100 action, abnormalities, accident, the factor at the 
time of trouble generating, etc. can be presumed, by analyzing the action log containing data which were 
mentioned above. 

[0100] The operating state which the robot 100 concerning this example can take is shown in drawing 8 As 
shown in this drawing, the robot 100 concerning this example can have the condition of calling it an active 
drive and a passive drive. 

[0101] In a passive drive, the command interpreted uniquely is received from a user, and actuation is 
performed only as directed, and internal chorus singers whom a robot 100 has, such as thinking and feeling 
do not function at all. 

[0102] For example, the walk pattern accumulated beforehand is read to the user directions "walk" and the 
drive condition of realizing walk actuation is equivalent to a passive drive. In such a case, although a robot's 
100 control function makes correction of the target orbit accompanying posture stability control, disturbance 
etc., neither thinking nor feeling acts and the dialogue with a user is not materialized (or it is unnecessary) 
[0103] Therefore, in the state of a passive drive, the inputted command is written in the contents field of a 
dialogue as it is among the action logs shown in drawing 7 . Moreover, since robot 100 self does not form an 
action plan, as for the action plan field, a blank or a default is written in. Moreover, the identification 
information which shows that it is in a passive drive condition is written in the mode-of-operation field 
Moreover, the implementation situation of operation computed based on the output of each joint angle sensor 
132 and the self-test results (failure situation etc.) judged based on each other sensor outputs are written in an 
action result and self-test result field. 

[0104] Moreover, in an active state, there is two kinds of operating state of the "dialogue drive" method 
which follows a dialogue with the user based on the input of voice, an image, etc., and forms an action plan 
and the "autonomous drive" method which does not depend on a dialogue with a user but forms an action ' 
plan autonomously. 

[0105] A robot 100 may be made to operate by either the dialogue drive method or the autonomous drive 
method, and may be switchable in between both drive methods in both directions or an one direction For 
example, it may change to an autonomous drive method according to generating at predetermined events 
such as elapsed time from the last user input or dialogue, or you may make it return to a dialogue drive 
method according to the next user input or generating of a dialogue. 

[0106] The operating sequence of the robot 100 at the time of an active drive is typically shown in drawing 
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[0107] At the time of a dialogue drive, if a user input occurs, while recognizing this, it will interpret as a 
command and dialogue management will be carried out. Moreover, authentication processing of the user 
who has a dialog is carried out through a fingerprint sensor or a visual sensor. Furthermore, an action plan is 
formed based on the external environment by which the sensor input was carried out with the contents of a 
dialogue, and the pattern of operation corresponding to action is performed. The activation situation of a 
pattern of operation is acquired by the sensor input. 

[0108] Therefore, in the state of a dialogue drive, the authentication information about the user who performs 
a dialogue with a robot 100 is written in user authentication information field among the action logs shown 
in drawing 7 . Moreover, the contents of a dialogue materialized among users are written in the contents field 
of a dialogue. Moreover, the action plan which the robot 100 determined according to change of the contents 
of a dialogue or an external environment is written in the action plan field. Moreover, the identification 
information which shows that it is in a dialogue drive condition is written in the mode-of-operation field 
Moreover, the implementation situation of operation computed based on the output of each joint angle sensor 
1 32 and the self-test results (failure situation etc.) judged based on each other sensor outputs are written in an 
action result and self-test result field. 

[0109] On the other hand, at the time of an autonomous drive, the user input has stopped, a user input is not 
received, therefore the dialogue with a user is not materialized. In such a case, a robot 100 does not depend 
on a dialogue with a user, but forms an action plan autonomously based on change and the feeling model of 
an external environment by which a sensor input is carried out. And the pattern of operation corresponding to 
action is performed, and the activation situation of a pattern of operation is acquired by the sensor input At 
the time of an autonomous drive, since the user who is on the outskirts is not command issue origin, even if 
he performs user authentication, he does not need to carry out. 

[01 10] or [ therefore, / that the authentication information of the user who is on the outskirts is written in user 
authentication information field among the action logs shown drawing 7 in the state of an autonomous drive ] 
- or they are a blank or a default. Moreover, the inputted command is written in the contents field of a 
dialogue as it is. Moreover, the action plan which the robot 100 determined according to change of a feeling 
model or an external environment is written in the action plan field. Moreover, the identification information 
which shows that it is in an autonomous drive condition is written in the mode-of-operation field Moreover 
the implementation situation of operation computed based on the output of each joint angle sensor 132 and ' 
the self-test results (failure situation etc.) judged based on each other sensor outputs are written in an action 
result and self-test result field. 

[0111] Even if you are in which drive condition which a robot 100 shows to drawings by analyzing the 
action log containing data which were mentioned above, please fully understand the point that a robot's 100 
action, abnormalities, accident, the factor at the time of trouble generating, etc. can be presumed 
[0112] It has explained in detail about this invention, referring to a specific example more than [addenda] 
However, it is obvious that this contractor can accomplish correction and substitution of this example in the 
range which does not deviate from the summary of this invention. That is, with the gestalt of instantiation 
this invention has been indicated and it should not be interpreted restrictively. In order to judge the summary 
[0U3] mVentI ° n ' C ° 1Umn ° f the Claim indicated at the beginning should be taken into consideration. 

[Effect of the Invention] As a full account was given above, according to this invention, the great action plan 
mold robot which follows a dialogue with the user based on the input of voice, an image, etc., an action plan 
can be formed, or cannot depend on a user input but can form an action plan autonomously can be offered 
[01 14] Moreover, according to this invention, abnormalities and failure can be generated during the period 
which performs "a dialogue drive" or "an autonomous drive", or even if it is the case where it is involved in a 
trouble, the great action plan mold robot which can study the cause can be offered. 
[01 15] The robot concerning this invention can attest the user who gives a robot a command. Or the 
description information on a user's face which cannot be attested, or voice or other living bodies is extracted 
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Furthermore, it records combining the contents of the command given by the user, and the activation time of 
day of action or action which the robot took according to the command. In addition, a robot's internal state 
and sensor input may be recorded collectively. Therefore, a user can study the cause of the abnormalities and 
failure which took place to the robot, and a trouble by analyzing the contents of record. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[ Drawing I [ It is a **** Fig. about signs that the leg formula mobile robot 100 with which operation of this 
invention is presented was viewed from the front. 

[Drawing 2] It is a **** Fig. about signs that the leg formula mobile robot 100 with which operation of this 
invention is presented was viewed from back. 

[Drawing 3] It is drawing having shown typically the degree-of-freedom configuration model which the leg 
formula mobile robot 1 00 concerning this example possesses. 

[Prawin * 41 11 is drawin 8 having shown ^ally the control-system configuration of the leg formula mobile 
robot 100 concerning this example. 

[Drawin g 5 ] It is drawing having shown typically an example of the functional-block configuration which 
records action record or experience information in the leg formula mobile robot 100 concerning this example 
[Drawing 6] It is drawing having shown typically other examples of the functional-block configuration 
which records action record or experience information in the leg formula mobile robot 100 concerning this 
example. 6 

[Drawing 7 ] It is drawing having shown the example of a data format of an action log. 

[ Pr e awin g 81 U is drawine having shown *e operating state which the robot 100 concerning this example can 

[Drawing 9] It is drawing having shown typically the operating sequence of the robot 100 at the time of an 
active drive. 

[Description of Notations] 

I — A head, 2 — Neck joint yawing axis 

3 - A neck joint pitching axis, 4 - Neck joint roll axes 
5 - A truncus pitching axis, 6 ~ Truncus roll axes 
7 - A truncus yawing axis, 8 - Shoulder-joint pitching axis 
9 - Shoulder-joint roll axes, 10 - Overarm yawing axis 

I I -- An elbow-joint pitching axis, 12 - Forearm yawing axis 
13 - A wrist joint pitching axis, 14 - Wrist joint roll axes 

15 - A hand part, 16 - Hip joint yawing axis 

17 - A hip joint pitching axis, 18 - Hip joint roll axes 

19 - A knee-joint pitching axis, 20 - Ankle joint pitching axis 

21 - Ankle joint roll axes, 22 - Foot (vola) 

100 ~ Leg formula mobile robot 

200 -- Thinking control module 

201 - Bus interface 

21 1 - CPU, 212 - RAM, 213 - ROM 
214 — External storage 

251 - Picture input device (CCD camera) 

252 - Audio input unit (microphone) 

253 ~ Audio output device (loudspeaker) 
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254 — Communication link interface 

300 — Movement control module 

301 — Bus interface 

311 - CPU, 312- RAM, 313 - ROM 
314 - External storage 351 - Attitude sensor 
352,353 — Touch-down check sensor 
354 — Power control 



[Claim(s)] 



[Claim 1] The action plan section which is the control device of the robot of a type which consists of two or 
more joint actuators, and operates according to an action plan, and forms said robot's action plan, The control 
section of operation which realizes the pattern of operation corresponding to the action plan determined by 
said action plan section by the drive of each joint actuator, The control unit of the robot characterized by 
providing the detecting element which detects the implementation situation of operation by said control 
section of operation, and the action plan by said action plan section and the Records Department which 
records a log including the implementation situation of operation by said detecting element. 
[Claim 2] Furthermore, it is the control unit of the robot according to claim 1 characterized by said Records 
Department taking the log of the contents of a dialogue while said action plan section forms an action plan 
according to the contents of a dialogue in said dialogue Management Department including the user input 
section which receives the command or data from a user, and the dialogue Management Department which 
manages a dialogue with the user based on the user input command or data through said user input section. 
[Claim 3 J Furthermore, said Records Department is the control unit of the robot according to claim 1 
characterized by taking the log of a self-test result including the self-test section which diagnoses said robot's 
each part. 

[Claim 4] Furthermore, said Records Department is the control unit of the robot according to claim 1 
characterized by taking the log of a user authentication result including the user authentication section which 
carries out authentication processing of the user who is around said robot. 

[Claim 5] Said action plan section is the control unit of the robot according to claim 1 characterized by 
forming an action plan based on the feeling model determined according to external change etc. 
[Claim 6] Said action plan section is the control unit of the robot according to claim 1 characterized by 
taking the log of a method [ in / it can operate with the 2nd method of operation which forms an action plan 
based on the feeling model determined according to the 1st method of operation which forms an action plan 
based on a dialogue with a user, or external change, and / in said Records Department / the action plan 
section ] of operation. 

[Claim 7] The action plan step which is the control approach of the robot of a type which consists of two or 
more joint actuators, and operates according to an action plan, and forms said robot's action plan, The 
motion-control step which realizes the pattern of operation corresponding to the action plan determined by 
said action plan step by the drive of each joint actuator, The control approach of the robot characterized by 
providing the detection step which detects the implementation situation of operation by said motion-control 
step, and the action plan by said action plan step and the record step which records a log including the 
implementation situation of operation by said detection step. 

[Claim 8] Furthermore, the control approach of the robot according to claim 7 characterized by to take the 
log of the contents of a dialogue at said record step while forming an action plan at said action plan step 
according to the contents of a dialogue in said dialogue Management Department including the user input 
step which receives the command or the data from a user, and the dialogue management step which manages 
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a dialogue with the user based on the user input command or the data based on said user input step. 
[Claim 9] Furthermore, the control approach of the robot according to claim 7 characterized by taking the log 
of a self-test result at said record step including the self-test step which diagnoses said robot's each part. 
[Claim 10] Furthermore, the control approach of the robot according to claim 7 characterized by taking the 
log of a user authentication result at said record step including the user authentication step which carries out 
authentication processing of the user who is around said robot. 

[Claim 1 1] The control approach of the robot according to claim 7 characterized by forming an action plan at 
said action plan step based on the feeling model determined according to external change etc. 
[Claim 12] The control approach of the robot according to claim 7 characterized by being able to operate 
with the 2nd method of operation which forms an action plan based on the feeling model determined at said 
action plan step according to the 1st method of operation which forms an action plan based on a dialogue 
with a user, or external change, and taking the log of the method of operation in an action plan step at said 
record step. 
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HUiEfBS *-r -y ^T-« 3 E^»f*g*© p 7* 5 C £ * 

mm £ 1 5 m#m 7 ict b«© p # >y h ©sij®^}£ 0 
[ifl&gi 0] set, itiBDsUy mjatjgsi— if 

*l§EMat 5a— »f ISIiEXx >y 

mmmzf- v zfT-ltZL— (f ig|I^©p £%z.£ 

*&%£?% mmm 7 ti Efc© p # h <Dwmm 0 

±Lxzc£ £?& mum 7 \z ie«© p # 7 h©M 
cia^i 2] mtfimtmxv-v-fxu, a— »ft© 

WgtS^^T^T l!lStiaii*i£T5^ 1 ©mff#s, x 

^© p 5 c k 1 5 li^JS 7 JcfEiJ© p 

v h©*ij»^ 0 

[|gH^©P*fflSiJiW] 
[000 1] 

ffrfFfSPjtf-y McfctK ^■^H^H©A7^ 
fcS-df < a— if ©*!»£ J; o Tff KlSt®*JiT fc D , 
a— tf A* t ^ s f @ »Wt ftMatmZiLX It *) 1 5 ff 

[0002] mk:S¥ L < *f£Bj^ r^isigifjj x 

h77;bfc#tii*nfc^&Tfeorfe^©iSH*^W 
[0003] 

cce*©a«3 *stweL<tt«awftitffl*ffl^rA 

Wo»^fc(K-&fcjfflB*fT3»KaEfll©«: £tt Tp^.y 
hJ P^-yh©»{4, X7»'R0B0 

tt, P4?-y h^nSL^46fc©(i: 1 9 6 0^*^6? 
*5A^ *©^<tt. I*fc43tt54«^3H©g«|fl:. 

ftif©«l8fflD^.y h (industrial robot) Xhitc 0 



[0 0 0 4] r-A^o^-y ho^tc, &%>ftj£<»m 

[0 0 0 5] Wd&T'lt, ^?-^M©«fc3fc:4£#fT© 
!&%©#&* #xXA^©iSf|:#^Lfc^'V h§yp# 

©#£M * - XA^i&ft^ LTff-i' >2nfc 
rAHJ&l SL<fi TAP^§yj OD^-y h (humanoid r 

obot) a 2, nij^ibD^^ Mcia-raw^w^Jis 

[0 0 0 6] WS£ftD#vh©JBj&DiofcLT\ & 
[0 0 0 7] £fc, IW^I&o^'V h©flil©ffl^£ L 

^Arat© r±t^j tt^ffliis&wenso £©a© 30 

Dtf^Mi, \l h&Z\,mj? h) &£©J±«W 
fcJtt © rgj t *B8:$frIM*©ftf* * * - X A ^SHt S£* 

[ooos] tae*©5c^«i*sii, ifjtfftjssftff 

£©!ISffia*@£W-e&9, U^(Om^=L-^(D^3).\Z 
HWt^tSctttT't^^o C©*§£, a— »f« 40 
(SI Cf&ft ») ig*4l^5W* WTttfi#T L * 5 

[0 0 0 9] CftfcJ#U Sl«ftn#.yM4, 

s^atf»ffSj»*fTa 1 1 tic, i&ft^^©0t^?ijt 

C££>nTt££&3 0 C©BS, a-Ht^^if^^gp^c, 
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^->*i(ftt5Li:tf-pt5. Sfc, a— iftt, -a 
[0 0 10] 

unit, ifgMiffla^jY/^ae-^m 

[0 0 1 l] CftlcffU ±t^§y^L<tix>^-xY 
>^>H©a^7h^ ±iELrcJ;3^. a— tf 
^P.Cfl|/7sWXii^Wa3vy K©##e>-r, 

f^-rs wigigfjj ^^Tofco, ssttta— tf^a© 

[0 0 12] LfrL£tf?>, ttfcWfta-lfA7J*A¥fR 
L £ 9 , naP >v h Jg @ ©©JS^fiiijil^.W^ Lfc 0 , fjfp 

[0 0 13] -!S©a«^S©ii^ AjjnvyHtcj* 

v > k* A7j mm?: l fzttic igmicmfimz l & 

[0 0 14] Ctilc^L, ±iE©J;^>5: rjsflSS»j X 
U rg^lgKu %f7o#J7<ou$y b(Dt§^ a— if 
Ajj ^^gPTOjc^-r § d jR >y h ±-e©»iRt *3b<« * 

p # >y h icmmc o tz<Dfr*$iwt s c t ttSi&TSt 
[0015] *»woewtt, ^^^^-©Ajjtc 

[0 0 16] *«W©3fE4*EIWtt, ^SSJBttJ Xt± 
Ztlz3bZ> 0 



(4) 
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CO 0 1 7] 

x-m&L z ft, nmmic o xmm a * f 7© □ # -y 

jz T § fT»ffrag|5X« Xf-y^i:, MISfftftttiSgB 
^SUBSIST' ^f-aX-dioBfttJ: DUSTTS 

^cj:si6^^»i*«m-r5«imasxtt^7 t 'y^ io 

£ , HfflE?Tf&ffiig|SX«: Xf^7*t±§ friftff® t IftiS 
fctfiSPXtt X t v 7 tc «fc S fSfl^SUHciRfc #£y a ^r*| B 
B-rsSESSXttXr £ll<I1-SC<t£#Mi: 

[0 0 18] *%W*c^«OiK'y HO{a»«fliXa:*tt 

Si— tfA^XfiXr-y^ tutea— ifA^SPXtt 
7, x «y 7£;J> L tc=L-ifXil nvyFXttf-^ 

^■A.T'fect^ 0 C©«k3ftlf^ SijI2fTitil+®g!5X(±X 20 
5o 

[0 0 19] Mfeo#>y h©&gB*^»fT5ga 

ffiiEiBgSPXfcS: Xt 1 v y« g E^BrlSII© n 
£SC£#T*£S 0 

[0 0 2 0] t5fBn#>y hHHfcjgSa— ifftffi 

tmm?%=L— mm%tz-$(biz^fx J -et>&^ 0 z.<d& 30 

a &if£\ SaESaSSBXtt^r- 'y tfMHEISII© 

[0 0 21] $fc, MIBtf»MffiXttXr-yyt4, n 
-if ^©W»K*tJi>TfTfttHi*4TS* 1 ©ifrft^ 
S, X&, ^gfcftHfcfcUTJfc^SftSSHiItT^ 

iiXr -y ^ifTftH-ilBgPXfi Xr-y WSIdffJSrS 

[0 0 2 2] 40 
[ftffi] *^tc^SD#'y Mi, M^Kntf'-y h<D 
«fc 5 ff*OffllffiM*«eBBW<ik:asf5-r€ c t^T* 

7?£S 0 £fc, *58lj|t«5n3KyK4, g^^iSfisa 

Tfc9, J^tli, P-^A^*fi£<mfl?i^«H- 
m%iLTtc *) ? s a t § 0 
[0 0 2 3] P#«y ht}g^*4iS.5a— iffcBHEfS 
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a— *f ^8 nfc#fr©rt« t , psK -y h fH^£ 
-a©p^"i:LTlB^LTfc< 0 *©{& atf-y b©l*l 
[0024] ay©rt$*UWf-r«i:i:tej;o 

[0025] s/c, s3tb#, &gfH#, h^7vmrc-&< 
10 if, o*yh©ffijoJIMI*Bl^5c:i:^-pta 0 fes 

[0026] *Kflo*6tfl!ioB« % im^*UiStt, 

20 [0 0 2 7] 

[0028] HiatfH2t«, *»woiysfc:«*n 

t$ws<d&% e> mm l rm=f-*7rs l t ^ s Q c ©bps 
8»njj?y hoob, rxr^j xti rxp^gyj 

<D*J]K.£r3 < a— if t Oj*Kt ± o TfT«ll+@i*3iT 
fcD, a— tfA^tasr (-rat>^, p.— tf*^3fi3i 
30 LT) P^-y h 1 0 0*<e»WKff«lttB*lETfc»)"r 

^?T^S^2£<DTKi:, &^©±Ki: % g 

g|5tT*^fi!c?ftS 0 

[0 0 2 9] tfe^OTKH:, Affigg(5i:, S&a@5i, 

fc J: o T^gp±^fc^MiJ^t TJiiS^ftT^So 
PSPtt, MBWffc«fcoT#»aii©l««±iaffi*fc: 

0 ais^ftr^So 

[0 0 3 0] ^g|5a~>y h|^(C«, 0 1 StfH2±T' 

« m x t ^ a ^ww&imm ^ ft r ^ s o c oftjm 

tt, iK&ijPtf-y h 1 0 0*«fi8f S#BB»7^^a 

©(fi, ®^fWII©MY>^-7i-X^ji35a^iS^: 

[0 0 3 1] 121 3 tea, *^)5S^Jl<:^sp^i|jp#-y 

1 MO O^Mf «Ba»iSJfi«fiE*«SWt^LT^ 
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3o NR&ftntfC-y h l oofi, 2*«®e 

su^an *^tr±f*^ mmttzmmtz 2 *ow 

[0 0 3 3] &»Ji, jSB§6!Sfcry^f*8 fc, M 
Ili5D-;H49t, ±§33-1*1 0£, Mfm^yHA 

lit, tuMa-m 1 ^nmmvv+m 1 3 

MHWn-;H*i 4£, s^lSfns, ^ 
Sfit3£ftT-&£ 0 fib, ^8Pi SOf&ftgtta, atfy 

[0 0 3 4] fta»tJ*y^-«T5 fc, 

d-;H*6£, f«*3-f*7fci^3gg|fi**rr5 0 

[0 0 3 5] TK*«j«-rsiEti*«(DWgBH:, 
W3-il6t KKSStr-y^Wl 7£, flSRIIlSD 

W2 0t, mu- ;1>M2 1 t, figg (ffifiXJijiW 
2 2£T-*MSft£o Af*OJE» OSg) 2 2*4, « 

*^0j£fc3WJi&£»|O#')/ h 1 0 0©JEj£li-tfDg 

[0 0 3 6] W±*«3gf titf, *£fflffl£ffi«Ni£g 

|D#7H0 0ifl!j:im £irT*3 + 7X2 + 3 
+ 6x2 = 3 2g4S%tt5<:iii:45. fiu WES 
»p#«v h l 0 OtfMLt 3 2 gSSlcPfi^^n§ 
IRT*ttft^ 0 aftfff • »^±®{&Jtt*fl=Jf»S*ft«^Ki6 

[0037] uplift o#»v h i o ot>m-D±i&<D&m 
ttkLrmmztiZo gffl©*W!±-efc#&»k#*& 

^5 ^e^Wifittfc: J* LT§af»»J»*fr a c t ft <5f cos 

LltjtJfiD'mA C+r-# • 7**aX-*£jgiBf 
ZZtkLfco ft&\ UPSCasK-y hta/liRlflift/J^A 
• 7^aX-*fc:|J£|LTii, 0Jxtf*|±jK<fiA 
^CMSftT^SWP 1 1-33386 IJHJJifl/f 

[0 0 3 8] i4t(i > «»j^^§i]^ija#--v 
h 1 0 0 ©aS->7f AM^iSfiti Lt l>§ 0 |1I 
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±mmmm^m'm^mmmm^j^-)i3 o o tr- 

[0 0 3 9] S^fiJ»ti/i-;|/2 0 0«, tii&SfiJBfr^ 
jaW«StBB-r*i«lil!yi*SlfT-rs CPU (Central 
Processing Unit) 2 11^, RAM (Random Access M 
emory) 2 12, ROM (Read Only Memory) 2 13, 

io atf , ftmzmmw c>\- k • x> * * • k 5 ^ 7 ft *?) 
[0040] JB#sy»t s; a -;u 2 0 0 t a, c c d 

(Charge Coupled Device) ^ft £©H{iA*^g 
2 5 Hf\ V^^ftif(D»A7JSfi2 5 2, T.M-tt 
^kcDB^mtimm2 53, LAN (Local Area Netwo 
rk:07r:LftV>) ft iffciH&LTPtfy h 1 OOft0» 
XxAi:7 ; -^x^tT^iifS-r>^-7x-X2 5 4 
Kk&M<DmWi)\ /U^>^-7x-X20 IMS 
20 TfSlBSftTVS. 

[0 0 4 1 ] &%MW*:i/z.-)l2 0 OTfi, HfiA73 
^i2 5 1 fr5Aa*ft*^r-*^ii^A*lS»2 

if left oT, BPS»i!in#-v h 1 o 0 ©St&oflffiHftB 

^i/*a-;I/3 0 OKftLTf^fcJEfiff So 
[0 0 4 2] -7f©Mijf|lJ^> ; a-;I/3 0 0!t P^K 
h l 0 0©^#^|i^«,^J^§c P U (Central 
30 Processing Unit) 3 11^ RAM (Random Access M 
emory) 3 12, ROM (Read Only Memory) 3 13, 

3 1 4T^?n§, SB^ISM^fT^Ct^tS 
®&L1tmmEM£W?*>Z. ^SPf2f8SE3 1 4tc 

§ ( TZMPj kl* y ^ff+OflcJK^tiSt-^yh 
*^Di:ftsjBSM±o^ocifcr«*!j, Sfc, tzmp 
fliij i:fi, Mx.ifn.f7 h l 0 0C0^fTi&ftl9K c f'ft 

40 iftc zmp &W}< w&zmmtz) 0 

[0 0 4 3] l|»t-;a-i3 0 0tC{±, P^<y h 

i ooo^#ic»«f**tiynoBBma*fi*iis[f 
&&pm*mt «»-b>^3 5 1 , £fe©£iso 

3 , M-y x U ft H©HM*«Bfi-r % WMSEiSft £<D 
#ffl©SK««, /U^>^-7x-^.'i 0 1 

[0 0 4 4] jlEd^ti/'a-^S 0 0m J&gffliB 
50 ; E^a-;b2 0 0*»6}g^*nfcffft*(*3S-r^<, § 
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CPU31 lit Jg#»ti>'a-/l/2 0 0*> 

5 * nfctf »k: j£ i; 1 5? - > * ^gpt ett^g 

3 i 4 3b»SSOfiUU x«\ fl8P«fcl&fM*->'££ 

/&lT3o LT, CPU3 1 1«\ Jg^Snfcf&fM* 
-^'fct ZMP (Zero Moment Poin 

t) fiii, ttswiis, HBBTkTOBarfiSs 
[0045] sifc, c pu3 i m, 5 1 

OW^ifKJ; Dn#'yM 0 0 Ott#gB#©ggJ*H£ 
f?**ffi-TSl:i:feli:, #S±te5t^-b>-9-3 5 2St>*3 

5 3offi*^(c«t*j^i(iip*t3fiBpxtt3iijiot^n 
[oo4 6] *&k:» mmum^^^~)i3 o o«, n 

[0 0 4 7] tiM5&m*Z?a—ll2 0 0 tmShfflMti? 
ft, S»U-^>^-7x-X2 0 l&tf3 0 1 
[0 0 4 8] *^SSfi?IJ{C^§ffiP^fjp^.y h 1 0 0 

j§bj§ c si? >y h off ftsffix ajsttHHB<oae»s 

[0 0 4 9] 0 5fc«\ KPj^«|D«-v h 1 OOfcfct^ 

|g^a >v * M©-M3r b» ltv* 0 mmiz^t d # 
•yHooii, ^WM£&£©A7Jk:S-3<a--if£: 
O^ggicM^LTffidftiii^iT-S r^ggfgijjj 

[0050] isiint;^ Hooii, a— tfiglEo 
fcfcfc. ffl&teyVl l lXtt«JE-fe>*M l 3©31S 
'>4< i:t»-75*fijffl-r5c 

[0 0 51] fflJi-fe>-9- 1 1 3W\ Wittf, HftSMC&iK 
D (Charge Coupled Device : Wnr&i^ 
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LT, IMMSP1 1 4ti\ 1 1 3frP>\j] 

[0 0 5 2] J&tt-bvy-l 1 Hi. El 1 ~Ei 4 frC 

ftfcBSWl^ttffifcJBttKJ&'V;, K*EHLTfc< 0 

* lt, i u±<Dm3.-*f<r>m%c*mmwn£ lt& 

10 6^U46ffWLTfe< 0 ^LT, ISIjE^M 1 1 2H\ 

wwimmm£t2>££&i^ &z>fr\:tbwmznrc 

EftBlfSo W3-1 8 9 8 0^fcfcJ\ 
t & S^OSHi^'- ZZ&ftt 3 gMli;* tut, 

isiim t & s m^oiifi^ atj^ -s mm atj^s t * 

^*ff 9 ^-Y ^OJgttHg^SEBtot ^ T ffl^ 2 ft T ^ 

So 

20 [0053] Jg^-fey-y- 1 i i smsiEMSP 1 1 2 
l 3&tfWuM!^l 1 4tt % ftlca— *f#2g;RL&^ 
fc, SliEjULSgpi l 2&tf l l 4 n.— tfoBSEIga 

^iivmsraffi 1 2 5 ic $5j^-r sct^TtSo 

[0 0 5 4] *H»J{C^§p^if|D^.y MOO 

a-^bY> K^f-^U^tSftftlc, ^ 

30 ggassp lis, an rasp 1 1 6 , mmm& 117, 

[0 0 5 5] mPWW&l 1 514, V-T^^ifO^A 
7JSP2 5 2 @ram9pT«g«;C P U 2 1 1 (Xtt 
^OffiO^SSfLIKHlSS) t<feoT«fi!cSn«o JL-+f 

r^ftj , roA^M-ij , iWj ^if©g^w 
^^cxt4isawft=iv> K*^TA*f5c twr* 

B^WMffil 1 5tti- »f©^^|g|iL, ?G 

10 [0 0 5 e] 1 6«, mm^yz- 

7x-X2 5 4 t, ii(i'<>^-7x-7 > 2 5 4^*T' 
5c^t53V> KXtt-r-^©atS*fT-5 C P U 2 1 l 
(X(4^0ffi0^g®iI(H]g§) £J:oT#|j**tt*o 31 
f!Y>^-7x-X2 5 4 {i, bluetooth^ 

©fteoMx-^M^^EJa-pn^ot,:/^!^ (p^fi' 
ttLTt^So a— iftt, CO«fc34tfI«iffl|5lU:'P, p# 

n>V-;UA^/LT, P^-y h 1 0 OfHfcfiarrSC 
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-+f- 37>F(t nvyKMWi i 9 (<::fc^Tf§f?R 

[ 0 0 5 7 ] Hftlgg^gp 1 1 7 CCDA^7ftHO 
S^A^gP2 5 1^ Wi|gIlMilpJi6^C P U 2 1 1 

(^^(Dimmw-mm^) \z&ixm$Li£tiz> 0 n. 
—fit, mmv^M*), yi^taHTavyF^ 
x.j3?%nttfT*izz> 0 mmmm&i i 7t4H{£igfas 
u ?p)tcwKWi 1 9 ttmm^Km^x 

X-vyVULTWM-Hffit&o fc&^tt, 10 

JO 0 5 8] nv>^i 1811, Si~H4ttt^KH 
✓IsL&l^, HK&l&Dtfy F l 0 OWW^ffiSBaif 

Mtf-mmm%R^m&mj£(D^y f, 

iHi»ff^©37>K^ 181? 20 

a— tf • rjv>F(4, 3T>F«HR»i l 9tm^TJ» 
ffl**ifc&, J*jgW.3|Jl 2 4K!K3MSft« 0 

ro o 5 9 ] ttmmm 1 2 4 (4, fi$$?i$ 1 1 

9»*"PSttflJ?ofc*|t^S©3V> F£, Efl^ljfityc 

ra-r s c t x\ 3.-v£<Dim<D*yT*x f zmm 

3BSOTSM 2 5t(E3aS-r5Ci:^T-#5o Sfc, *rts 
SSSPl 2 4ii, a— tfA*»cj£i;fcig*«»^ffi 30 

a^iwi 2 1 m&x°ftm\W8ki/7.7-L (MAifa 
— ma*) ±fc3>y-/i/tt}2j (x«gu i^^ijfflL/c 

[0 0 6 0] ^^figgPl 2 0(4, ^tf-*4ifflD^ 

tatimm 2 5 3 1 , ^s^/saagnr^a c p u 2 1 1 

(X«*©fl!!©8W*aJIlHll&) i:iot«?n§ 0 £ 
fc, iifg$iJMi 2 1 (4, jifI-r>^-7x-X2 5 4 
iifa-f>^-7x-X2 5 4«*"Z?S3ftl-«3v> 
F*X«7-*-*£>i&JI*fi : 5CPU2 1 1 (Xti-fi&ffi© 40 

[0061] gm-tyv 1 3 0, Mfi-b>-9- 131,1 

fff&g-feytf 13 2, tm-t>V 13 3, ^-fey* 1 3 

4 , masMsu i3 5,m, mmf&mmmm 1 3 e 

(4, od?«y F 1 0 0om?X(4SKa^*©Sfl:, Sftffcg 

[0 0 6 2] g^-fey* 1 3 0 (4, El 4 if>04?flB#R* 3 
5 4tC*H^L, P#'V F 1 0 OtD&ffifcft-rs&f^ft 

mtscims, sfc. Hafts-fey-* 1 3 2(4, 

&MIti7^5^x-£ (0 3*#lOCt) I5(c§ef2£ 50 
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nfcisitex>3-^fcffls-rSo mm-tyy- 1 3 

3 Si, JE»tES*tifc»Jft«SB-by9-3 5 2, 3 5 2 
Icffi^tZo Sfc, 2j-fe>m 3 4(1, Ef 1 ~EI4 tc« 
EfeL&t/^, o#y F l 0 0<D£#©&g|5fik:gBtS£ 

ftTt^SfWF 1 1 -3 6 6 1 7 5^«BSfC«, 

gPp n n©ff^T-D#-v b<D£%&mzmm?&n twx% 

[0 06 3] mWMmffl 13514, EI 4 ff 3 
5 4(C*§M3U P,1?y F 1 0 Q<D±mMT*$>2>f^y7-V 

t^*o sfc, ii^j^ffigpi 3 6«, mm^y*- 

7x-X2 5 4 t, ilfi^>^-7x-7,2 5 4^*1? 
^-TSnv^FXttx-^O^g^foC PU2 1 l 

[oo6 4] cn54r>-9-tST«S5nap*y f i o 

OO^SBfifrfttf-ilglSl 2 7&tfga^WfSPl 2 9 

[0 0 6 5] frllltf-iaaJ 1 2 7 tt, JtigMSB l 2 4 t 
*5ttS*f,^03>7 1 *X F^, §-b>«ti?a-;l/ 

1 3 0-t*ji>T«SSnsn#ry F 1 0 OOflUXtt 
«*4^tfier3T, a#«v F l 0 Q<DfiM[m*iLX 

*^S8$)JT*(4, frS&ff-jSfgp 1 2 7(4, fri&ffB© 

rts*ftaiiasga!gp 1 2 5 tisaf-rsc twxtz* 
[0066] mftumm 1 2 8 «, 1^11®^ 127c 

(H3&O t 0 4*#^ocii:) tWLTIsrtEig^ 

xttSB^snaa^y f 1 0 0 (Dftmtf^mz ti%o & 
mBTt^oLjL-ZiDmifctiLWit&Mmm-tyyi 3 

2 (c «t o XMfe s n, 7 w - fv < y ^ ^fy a c t ^ 

[0067] nf^sjm 1 2 s a^^*nsfri& 
» LTfc^Tt,j;^ 0 mgvms, mmm^i 28 

(4, frfMH-SSPl 2 7^5lr»f»-&3b««fT*ti«i:, c 
^f)^6 2-9 7 0 0 6 F^KIfctt, ^.glfiS 

titc#fr/< Z- y • r- ^ *J-H i^l^T', IfiHW^D ^ 

^^(co^ <- c t ^t* 1 5 zmm&fi p # >v f Magi 

[0 0 6 8] gE^gPl 2 9(4, $m*y-t}-l 3 0, 
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fifi-b^U-i 3 l, mBftfe-ty-V l 3 2, gfeHzy+r 

i 3 3, ;frt;/-9- 134, mmmmti 3 5, aiiftitP 

nfcfrftc^fTttia, SAW**, p* 

So Sfc Mt'li, gB»K0l 2 9 a, §3t£ 
Vr£££fll«iS&«ffi l 2 5fc|£2tf scfctf-e* 

So 

[0 06 9] fifflisswasp 125a, *»*»5ttT© 
«F«i*^-ra<:i:A^#So 10 

(1) siE&affi 1 1 2atf/xtti 1 4 3^^©n.— »f 

BSEtiHB 

(2) wiswagM 2 4tfet^T«a!*ti*»ert$x 

a«#r©P— f • pv>K 

(3) J*BrtSXtta— «f • *WXI4«« 
©£{fcfc Hit «t o Tft^^nSffilfHiS 

(4) gs&wan 2 9tfc^Titif^nfdTiiitii<D 

[0 0 7 0] ^LT, titM2imHg|5 12 514, 
©7 s -* ^£fcl$f+l 2 6frZim2tl 20 

SIH£B#$iJt l>o L J: IC LT, r^B/fflgj Xtt mm 

[0 0 7 1] frftn^tt, 0iUt4\ ffl^Sie^M^ 1 2 

* t 'J • Xf7 ^©J: o &*- h U S?a£T?ffiftRrai4 
ISffiSIBJcttiWbTio^tf, ?rz?P#-v h 1 ooi^ia 
*5 ft LT^gp©It3?«*>^-r A±T*fr»IP if&mttjBM 

[0072] sfc, Mfatsiiwagp 1 2 5 tfs^^ns 30 
mmmm_i xa rmmfiu a, Mi^ffl^Ti 
^Tsct^-ptSo &s^a, smatasi 1 

*fl-Utn:J?«y h l 0 0i:©^|g(cj:0?l^tb-rc fctf 
[0 0 7 3] 06fc(4, W3&9ha$y b 1 0 0103^ 
^p«v^^tWrSfte©M*Hft?LTl^o RHtc 

3.— »fi:©wfiiciifleLTfTtt9Hii*5iTs mmm 

®JJ Txft©^, 25>{c, a— »fi:©WISK*5-ril» 40 

[0 0 7 4] BPSCSffflnsfEy M 0 oa, p— IflgffiE© 
}gtt-fe>+M l tXttfflJK-b>-9-l l 3©5 5 
'>a<l:t-^*fiJJ ! l'J-rSo M'M-b>+M l 3(4, 00* 
(4\ S/jffilCfSili^nfcC C D (Charge Coupled Devic 
c : MK'i^#?) **7&£©S!iimW3£il!t2 5 1 * 

Kjffl-rscttf-pffs, sfc, m&*yyi 1 ia, m 
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LTfc< 0 

[0 0 7 5] fg^-tr^-y l l l Ktf88EftUi$ l l 2 
a, Jg«l3!a^*^W»cS*Lfta— IfKJ* LT.J: 9 IE 

1 3&tf|gMag|5l l 4a, ffilca— •fffgjfcLftt^ 

tftf&S^I©?-5£^M!UI*fir3o £fc, IgtEMSB 

i i 2rxii 14a, a— ir©K«E«s**«*BSEa«a 

SB 1 2 5£l£ES-t£ 0 
[0 0 7 6] Sfc. W£&WlU#y b 1 0 0a, P— if 

*^v>K*A*-rsfc«>k:, grami 15, mm 

«Wl l 6, MfcBMSl l 7, nvy^i lstt,^ 

[0077] ^imsr 1 1 sa, wr&muwx 
5 2 ts mpmmwmmft c p u 2 1 1 ex 
tt*©fft©MlMy!i5i8) t^oTfiidtsnSo p— tf 

tS^V^TP^y h 1 0 0t^t53v>F*lB • ft? 

[0078] iifisij«i 1 6a, iifg-r>^- 

7x-X2 5 4 t KXtt-r-*©$aS*fT 

•5 c p u 2 1 1 (xa*©ffi©ra«ia!ioi») (a^t 
«^ni. 0 i^>^-7i-^2 5 4 a, mm?- 

^HiSlST'^^iW* (tflJaJf/S-y^i, • p 
y¥a.-ti) ^sgiLti^. SHBffllfiiSiSi 1 6 

t4, p.— yimm\5\mm&T<mkT%3-?y b-*%mm 
m~?&o §m?tifcPv>Ka, 1 9 

So 

[0 0 7 9] iSftlS^gPl 1 7a, CCD*^7ft^(0 

mmxt}^2 51 mmmmmmmrji c p u 2 1 1 
-ifa, ti^ffiD, ^xx^-v^if©^r*p- 

■9 s - PvyF^Ati^nsi:, BftSNffii l 7acti 

*mmmm u s e t p v > k^hrsb 1 1 9 aigii^^ 

[0 0 8 0] 1 sa, W3&9bu#>y b l 0 

mm. xa, u^-h-p>hP-^ (h^l4^) t 

^&^©P^>Kftif*PV>^l 1 8±T'fT^jAty 
iltA^^Sc PV>^"1 1 8±TA7J^tl/cP-1f • 

pv>na, 37>k»i i 9tc^^T^!R*nfc 

ft, W.iS^I'l!.^ 1 2 4(DfjS?nSo 

[oo8i] w.ts^iii!^ 124a, ^-?y mm® 1 1 

+ S i: i t, icim p h ©|^$% 0 § „ wisra 

asi 2 4 a, p.— «f fcowerts^tiMHiHfiwaw 1 2 



15 

CO 0 8 2] g^lfigpi 2 0(4, Xt?-£ft£©^ 

w^/gS2 5 3 i^^jaasrfiiftc p u 2 1 1 

(Xtt*Ofl!!0»J|[ftyiigiS) tC«toT*J5E5tlSo $ 
fc, ii^JffilSPl 2 1(4, }§f§Yy#-7x-X2 5 4 
fc, if^>^-7x-X2 5 4g^T'5M53V> 
KXttr-dnojgyifcft-S CPU2 1 1 (X(4?©ffi<D 

[0 0 8 3] §6»-fe>-9-i 3 0, rat-fey+J- 1 3 Ml 
flSftfi-b>* 13 2, gftti-b>-9- 13 3, 1 3 

4, TOeaspi 3 5, mtEM'm&ffi 1 3 6(4, 

v h 1 0 0©^Xtt«W^*©S{fc*fcW"rS;fcii>© 
[0 0 8 4] gtMrviJ- 1 3 0(i, d^-v h l 0 0O£S 
2*4, &HW7*f-ax-*fc|cEK£nfc@lEx>ri 

-^t«a-r«o gas-try* 1 33(4, jEastE 

^?n/cgtt!l6l^-fe>-9-3 5 2, 3 5 2\Z*mtZ>o S 
fc, ^^13 4(4, ptfy h l 

ai-r^o rasaas 1 3 5(4, h 1 0 o<&±nig 
t^o $fc ietgwmiisi 3 6(4, jif§^>*-7 

x-7. 2 5 4 <t, ilfinvy FXfctr-*©JGS*fT5 
CPU2 1 1 (X«*©fl&©SOM!!JilUK) ti^ti 
ii^nv>KX(4ii{i7 s -^^J;SnWX(4S 

[0 0 8 5] ctiet^s-ps^nsn^y h 1 0 
oaamHBg&ffflbftiiiiffl 1 4 ostfiB^Wrgp 1 

2 9(C«tl§ 0 

[0 0 8 6] HWffiftW-iSgpi 4 0(4, ±XH*>vmfr 
£©A*ix-*ft if (cg^j < m88ft0&{b& H tcS-df 
i^T, MtxVl/^Mvi/y 1 4 l rtOS'lf ^E-fvl/^ 
^TSiii:?, SfliWJCDjj?^ h 1 0 0 OfflWH-®*fi 
TT, ftfftPjfllffl! 1 2 8t#LTffft©H^*lflt«W 
§0 SftfTfftltiSSP 1 4 0(4, (ff&ltiSOF*J§* 

ffSfBSMSP l 2 5 tGiSI-rsc fc*«"Zf**o (ft 
*s, *ffliSAfc:HfcraStttV*1WF l 1-341 

3 7 4^0^»(<:(4, a^tceH-rSiS-(S*^tr;l/* 

[0 0 8 7] IlifrMlJM l 2 8(4, lvm^T»aiJiffi 1 4 
zl x- * (eft LTlEifiJg^#M^ft ifoymim^ 



0) ftBB 2001-277163 

16 

OO^tT^Hfl^nSo *IBW7^f-aX-^©|5HE 

tt«tt«B»ftS4*>-9-l 3 2(;:4.oT$J£2ft, 7^ 
[0 0 8 8] I&ftSiJffllgB 1 2 8 (4, j&MW*;£ft«fT» 
LT t> «fc t ^, ^fr^Ofl&O^gfcl&ffr ^ - > 

[0 0 8 9] gai^gfli 2 9(4, g^-b^-y- 1 3 0, 

?MS-b>+M 3 1, 1 3 2, 

10 1 3 3, imyVl 3 4, SMIgRl 3 5, If|(j|l 
fc«l 3 6 420*>iJ-«©fcttttt**Sk:, ftiS^ 

BP 1 2 9 (4, SS^»f^*M«fHSraSP 1 2 5 (Zfe 

[0090] mmmwsffi 12 sit. g-gi^wT© 

( 1 ) l2tI£LIIg|5 1 1 2 &1>VX(4 1 l 4 
KKEflHH 

20 ( 2 ) ttfSWUgP 1 2 4 Kfe^TSSafnSJWSrtgX 

(3) flHut^nctaeoTafljfTaffwspi ioa^ 

LfcfrHlSfiiSi 

(4) ga^ifSPi 2 9(cfc^T^?n/c»3tiii© 

[009 1]? LT, ffiffltBS^aS? 1 2 5(4, CtlS 

30 [0092] frftp^tt, «ij*.fcr, tsfBia^igp 1 2 

5©P-*;b- ^^'JX(4a-*;l/.x^x^ (0^L 

[0 0 9 3] Sfc, m-OasRv h 1 OOA^Ii 
ft if © Xti (c g-i - < a— tf t Oi*KKiifle L Tff KltHi 

to mmcftmamttiLTz r&wmm ttvww® 

hT§±I^t(4, fTAftfitfttftSax 
ii-T§4:^(cLTt4:(/> 0 

[0094] m$sie$iwng[5 1 2 5(c«???n§ rtj®, 

» X(4 rufj^jftj (4, HM'Jt t ! /^JIJt>T^-r5 
Ci:tfr*f?So £^(4, 1 5ftif*/>L 

fcP*->v h 1 0 0 tommc «fc D&'ittH-rc t^-e* 
§ 0 

[0 0 9 5] 0 7(C(4, fflftP^O-r--^ • 7^-V-y 
h^iJ^LTl^c /WKUtCT^-TJ; ^tc, 10©P^(4, 
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7 j -)l F t , frftlg Jl&tf i BIMIili JI7 * F "Z? 
CO 0 9 6] B#^j7^-;UFlC(i, B#ft 1 2 6jb^^ 

im-ty-vi i i Jkvmsmm^ i i 

2, mt*.yV\ 1 SR^'IgglM^l 1 4 

t ± o TBK*tifc^i-1ffl|«***#iiSnSo Ma 

[0 0 9 7] fr»SHi7-r-;l/Ffc:{4, fffilt® 
SB 1 2 7Xtti»tT»fffBSSl 4 0KJ;oT4TSnfe 
trlWH-H^»f?iiSnSo Sfc, idftt— F - "7y— ;U 
KKtt, S«EOo#-; h i o 0 Wl&ltWttfS^i&X 

[0 0 9 8] fT«IS«&tfaH^»rtt*7-<-;U 

[0 0 9 9] ±&LfcJ:9ftr-**#tyff»ny*j!p 
fftSCitioT, P#-y h 1 OOOfffft-^, gfj?, 

[oioo] 08 tea, *HI«t:l5n#7 h l o o 

J: 3 fc, *£gffi09£fl& D>f7hl00[±, flgfUBft t 30 
fill z. t ft? t So 

[0101] SUITS, n— »ffr£-glJ*j£ft?lR£ 

mnu h i o off&os&^mi&EtDpitm 

[0102] m*.i£, rmn £^D=L—vmnkK.ttL 
T.joZfrv&wmznrzWn^z-yitmfriiiLT, 

<aj: s d#>v f i o ocommmmz. 
apj»wa4^t#^BflW!fiiafl!)i>jEftif*ff-5^, s 40 

^MiiftfflW, n— !f£©#S§ttJSiL4^ (X 

[0 10 3] LfctfoT, Sft«ft«J6Ttt, B7fc* 

^nv>F^ ; f©SSaf^iiSn5o P^-vFl 
oo£-i^tifT» f ttiffii*iT4i>o^ fTSftiH-iS7 ^ 

f • 7 4 ~>i f £ a, swijmiT-s 

a^Wf*g*7-r-^Kfc{i, #ii»ft«-b>-9-i 3 2© 50 
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[0 10 4] gfc, fl&IStfagT'fci:, ^j&Wfc&JfOA 

frftttoiifcji-cs rg^jas&j 2ffi9©I&{t 

[0 10 5] h 1 OOfct, J*aSB»^XttSft 
Bft3^0t^tl3!p-^0*-pii(W § «fc a (C LT fe «fc 
t^U WBft^M*5R^lfllXJ4-*lRl-e»»)SAnifll 
T?$oT6±l\ 0J*tf, ll©a- «fA*Xtt#Kfr 

[0 10 6] m9 ttt, BgiSM^C^t-SD^-y F 1 

[0107] ^eif^ictt, c 

n^rlSli-r * £ fc n v > K L T SHR L TirtSBS! 
>^AtI^n/c^W^li^c|£^^T^m ! ^tiii^Ii:TT, 

[0 10 8] Lft^T, J*lfSffi«F«JiT'tt, 0 7til^ 
ffTija?"©?^ a— »fSiEflHH7^-;l/FfcH:, a 

h 1 0 0 t<DMm*fi 5=1— »r jciB-rssii'itai^ 
®7^-;bFica, f 1 0 otfmmn^n&m 

fc, Slft^-F • 7^-;UFtcti, jWBHtt«ffiT%« 

[0109] gftsm^u, a— tfA*j#i£*e 

•y h l 0 0(i, a— !ffc©«KtlB6-f, -fe>+)-A^^ 

tim^mmcDmttmm^Mzm^rmmzn 
m\m*tLT% 0 fit, tT®j{c^js-rsi?i^^-> 

>F^fT7cT*a^l>OT% a— !fffiaE*tToTt)(7t>a ' 

[o i i o] LfttfoT. a»Ki&tt«-ea, 0 7^-r 
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§o Sfc, frlli*S«atfea»KIS«7^-;l/FK:U:, 
Mag^g-fe^ 1 3 2©tti*jfc:S-c$^Tmtti£tt7"c$j 

[0 1 1 1] ±$LfcJ;9&7 : -*;£^tmf&P^£#P 
^ff^C tfCckoT, P^«y F l 0 0tfm8fc7Kt^? 

ti<omW}vmx*&-?xt>, f i oo^tTft^ s 

CO 1 1 2] [Jiffi] JL^±, ^£©g|B601J£#0gL&tf 
CO 1 1 3] 

W±»K L/fc «t 3 fc, *58UHfc:«fctu£ 

iditMixT fc t> -r s c £ # t-# s, mtitcnm mm 
Co i i 4] $fc, *awtititf, r^tsSffttj xa 30 

5 C i: #T* t 3 , MnfcfTfttf-aS! n,f-v h £ jf&T § 

co 1 1 5] *fzwicm%utfy hn, bizma 

T* * * I u- »f©ig § v Hi foffio^ftlc^a 

tf^*ttto^rao set, a— «f*^#A&nfejt^o 

D-Ky Ftf^Kl£DT^ofcf7i&Wl&©f| 
fTl$#J<t£$§#£fcH±Tie$rf So ^©ffi, D>f7 KD 40 

fttf^T. ^— Wis IB»rt«*«5#frscttJ:o 
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0 e l rcm-izmtt wx°& So 

CE1 2] *»E©£ftlC«SftSW3#»D;j;»/ F 1 0 

cm 3] *H)K60ij{c^sKi^iiD#^ f i o owmm 

CH 4] 4^m(c^«|ip^iia^>y F i oooHty 

Cm 5] *HttMk:flbSIP^ibD^'> h l oot4s^ 
Tm^SLXttJi»ia$B*fBgr 3»8I7p 7 ZMtiKD 

[06] *H)5S^Jlc^§W^iiD*'>v F 1 0 OKfct* 

[07] ffftP^OT 5 -* • 7*- V«y FW*^LfcH 
T'&S 0 

Cm 8] *^«gfi»Jt^5D^*y MOOtf^CfctfT' 

cm 9] tgi&HijBt^&tfsatf-'y f i o ooirfts/- 
> w*c * l mi? % s o 

1-519$, 2-W8B8P3-H 

3 -gmm^vj-w, 4 -tmrnu-jim 

5 -*#try 6-f*#P-;HMi 
7 -ttftaHBi, 8 -«7MI6ritf 
9-HBBain-;I/tt. 1 0-_UR3-*l 

1 1 v^tt. 12-MK3-W 

1 \ 4-^Mmwv-jm 
1 5-m 1 6—jRBaas3— wi 

1 9-Hfgafijsif y^-fu, 2 

2 l -ElIBn-Wl 2 2 -JESS (JEfi) 

1 oo-KP^i&p^'v h 

2 0 1 -AX^>^-7x-X 

2 11-CPU. 2 12-RAM. 2 13-ROM 

2 1 4-nmzmmm 

2 5 l"«At)^B (CCD*^7) 
2 5 2-»A*»ll (vY*) 

2 5 3-^m^g utr-*) 

2 5 4-ilfr<>$r-7x-X 

3 0 0-a««iJ»ti?a-;I/ 

3 0 1 -^U-r>jr-7x-X 

311 -CPU, 3 12-RAM, 3 13-ROM 

3 1 3 5 1 ■ 

3 5 2,3 5 3 

3 5 4 -fg£M0£Hf 




CH9] 




(so int. ci. 7 mmi 

C 0 5 B 19/4155 

G 0 6 N 3/00 5 5 0 



F I 
G 0 5 B 
G 0 6 N 



19/4155 
3/00 



S 

5 5 0 E 



3F059 AAOO BA02 BB06 BC07 CA05 
CA06 DA02 DA05 DA09 DB02 
DB09 DCOl DC04 DC07 DDOl 
DD08 DD11 DD18 FA03 FA05 
FBOl FB13 FB17 FB29 FC02 
FC03 FC06 FC13 FC14 FC15 

5H004 GA26 GB16 HA07 HB07 KD56 
KD58 

5H269 AB21 AB33 BB03 BB09 BBll 
CC09 GGOl MM07 PP02 QE34 
QE37 

9A001 BB04 BB06 DZ15 FZ04 GZ09 
HH05 HH19 HH34 HZIO HZ15 
HZ20 HZ22 HZ23 KK32 KK37 
KK62 LL06 



